Curation
Curation¶
Note: make a copy of this notebook and run the copy to avoid git conflicts in the future
This is the second in a multi-part tutorial on the NWB-Datajoint pipeline used in Loren Frank's lab, UCSF. It demonstrates how to curate the results of spike sorting.
Finish tutorial 0 and tutorial 1 before proceeding.
Let's start by importing the spyglass package, along with a few others.
import os
import numpy as np
import spyglass as nd
import warnings
warnings.simplefilter("ignore", category=DeprecationWarning)
warnings.simplefilter("ignore", category=ResourceWarning)
[2022-08-01 17:21:49,989][INFO]: Connecting zoldello@lmf-db.cin.ucsf.edu:3306 [2022-08-01 17:21:50,035][INFO]: Connected zoldello@lmf-db.cin.ucsf.edu:3306
/home/zoldello/anaconda3/envs/spyglass/lib/python3.9/site-packages/position_tools/core.py:3: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated. from scipy.ndimage.filters import gaussian_filter1d
# We also import a bunch of tables so that we can call them easily
from spyglass.common import (
RawPosition,
HeadDir,
Speed,
LinPos,
StateScriptFile,
VideoFile,
DataAcquisitionDevice,
CameraDevice,
Probe,
DIOEvents,
ElectrodeGroup,
Electrode,
Raw,
SampleCount,
LFPSelection,
LFP,
LFPBandSelection,
LFPBand,
FirFilterParameters,
IntervalList,
Lab,
LabMember,
LabTeam,
Institution,
BrainRegion,
SensorData,
Session,
ExperimenterList,
Subject,
Task,
TaskEpoch,
Nwbfile,
AnalysisNwbfile,
NwbfileKachery,
AnalysisNwbfileKachery,
)
from spyglass.spikesorting import (
ArtifactDetection,
ArtifactDetectionParameters,
ArtifactDetectionSelection,
ArtifactRemovedIntervalList,
SortGroup,
SortInterval,
SpikeSortingPreprocessingParameters,
SpikeSortingRecording,
SpikeSortingRecordingSelection,
SpikeSorterParameters,
SpikeSorting,
SpikeSortingSelection,
SortInterval,
SpikeSorting,
AutomaticCuration,
AutomaticCurationParameters,
AutomaticCurationSelection,
CuratedSpikeSorting,
CuratedSpikeSortingSelection,
Curation,
MetricParameters,
MetricSelection,
QualityMetrics,
UnitInclusionParameters,
WaveformParameters,
Waveforms,
WaveformSelection,
)
# Define the name of the file that you copied and renamed from previous tutorials
nwb_file_name = "beans20190718.nwb"
filename, file_extension = os.path.splitext(nwb_file_name)
nwb_file_name2 = filename + "_" + file_extension
First, make sure that the results of your sorting from tutorial 1 are stored in SpikeSorting table.
SpikeSorting & {"nwb_file_name": nwb_file_name2}
| nwb_file_name name of the NWB file | sort_group_id identifier for a group of electrodes | sort_interval_name name for this interval | preproc_params_name | team_name | sorter | sorter_params_name | artifact_removed_interval_list_name | sorting_path | time_of_sort in Unix time, to the nearest second |
|---|---|---|---|---|---|---|---|---|---|
| CH6120211203_.nwb | 1 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | clusterless_thresholder | default_clusterless | CH6120211203_.nwb_raw data valid times_1_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_1_franklab_tetrode_hippocampus_d02e4cc4_spikesorting | 1652421556 |
| CH6120211203_.nwb | 1 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | mountainsort4 | franklab_tetrode_hippocampus_30KHz | CH6120211203_.nwb_raw data valid times_1_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_1_franklab_tetrode_hippocampus_3ce905c4_spikesorting | 1652390802 |
| CH6120211203_.nwb | 2 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | clusterless_thresholder | default_clusterless | CH6120211203_.nwb_raw data valid times_2_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_2_franklab_tetrode_hippocampus_3236edfe_spikesorting | 1652421602 |
| CH6120211203_.nwb | 2 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | mountainsort4 | franklab_tetrode_hippocampus_30KHz | CH6120211203_.nwb_raw data valid times_2_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_2_franklab_tetrode_hippocampus_175d4957_spikesorting | 1652391136 |
| CH6120211203_.nwb | 3 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | clusterless_thresholder | default_clusterless | CH6120211203_.nwb_raw data valid times_3_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_3_franklab_tetrode_hippocampus_0aa65847_spikesorting | 1652421651 |
| CH6120211203_.nwb | 3 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | mountainsort4 | franklab_tetrode_hippocampus_30KHz | CH6120211203_.nwb_raw data valid times_3_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_3_franklab_tetrode_hippocampus_b3b560dc_spikesorting | 1652391465 |
| CH6120211203_.nwb | 4 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | clusterless_thresholder | default_clusterless | CH6120211203_.nwb_raw data valid times_4_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_4_franklab_tetrode_hippocampus_a248a29c_spikesorting | 1652421729 |
| CH6120211203_.nwb | 4 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | mountainsort4 | franklab_tetrode_hippocampus_30KHz | CH6120211203_.nwb_raw data valid times_4_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_4_franklab_tetrode_hippocampus_52ff93db_spikesorting | 1652391770 |
| CH6120211203_.nwb | 5 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | clusterless_thresholder | default_clusterless | CH6120211203_.nwb_raw data valid times_5_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_5_franklab_tetrode_hippocampus_c75bf7f7_spikesorting | 1652421799 |
| CH6120211203_.nwb | 5 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | mountainsort4 | franklab_tetrode_hippocampus_30KHz | CH6120211203_.nwb_raw data valid times_5_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_5_franklab_tetrode_hippocampus_962a4aa4_spikesorting | 1652392074 |
| CH6120211203_.nwb | 6 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | clusterless_thresholder | default_clusterless | CH6120211203_.nwb_raw data valid times_6_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_6_franklab_tetrode_hippocampus_11b507a6_spikesorting | 1652421841 |
| CH6120211203_.nwb | 6 | raw data valid times | franklab_tetrode_hippocampus | Michael Coulter | mountainsort4 | franklab_tetrode_hippocampus_30KHz | CH6120211203_.nwb_raw data valid times_6_franklab_tetrode_hippocampus_ampl_1500_prop_001_5ms_artifact_removed_valid_times | /stelmo/nwb/sorting/CH6120211203_.nwb_raw data valid times_6_franklab_tetrode_hippocampus_ca67c53d_spikesorting | 1652392496 |
...
Total: 4702
sortingview web app¶
Currently (June 2021) the manual curation of spike sorted data is done on the sortingview web app. To proceed, click on the link printed at the end of spike sorting in tutorial 1. Alternatively, run the following cell and click on the URL output.
# workspace_uri = (SpikeSorting & {'nwb_file_name': nwb_file_name2}).fetch1('curation_feed_uri')
# print(f'https://sortingview.vercel.app/workspace?workspace={workspace_uri}&channel=franklab')
This will take you to a workspace on the sortingview app. The workspace, which you can think of as a list of recording and associated sorting objects, was created at the end of spike sorting. On the workspace view, you will see a set of recordings that have been added to the workspace.

Clicking on a recording then takes you to a page that gives you information about the recording as well as the associated sorting objects.

Click on a sorting to see the curation view. Try exploring the many visualization widgets.

The most important is the Units Table and the Curation menu, which allows you to give labels to the units. The curation labels will persist even if you suddenly lose connection to the app; this is because the curaiton actions are appended to the workspace as soon as they are created. Note that if you are not logged in with your Google account, Curation menu may not be visible. Log in and refresh the page to access this feature.
